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Description 

5 Method and device for detecting an object or a person 

The present invention relates to a method and device for 
detecting an object or a person, especially in the passenger 
compartment of a motor vehicle. 

10 

The known devices for image data processing have an image 
capture unit with at least one image sensor which records an 
image line-by-line, for example, and immediately communicates 
the image data which has been captured line -by- line to an 

15 image control unit, via an image data bus. Such an image 
sensor is also referred to as a "Rolling Shutter" image 
sensor. The image control unit generally consists of an ASIC 
or an FPGA, which processes the image sensor data in real 
time, and via a 32 -bit bus stores it temporarily in a buffer 

20 memory (e.g. DRAM). A microcontroller controls the ASIC and 

the data transmission in the image data processing system. In 
addition, the microcontroller analyses the image data held in 
the buffer memory. Finally the microcontroller, as part of an 
electronic control unit (ECU) , for example for an occupant 

25 protection system, executes algorithms to analyze the image 
data which has been received. 

For the case exemplified of line-by-line image recording from 
the image sensor, only a limited time period is available in 
3 0 the case of moving images for making the recording, because 
otherwise the recorded image will have been smeared" . 
Consequently, all the data for the image must be transmitted 
within this limited time period, which results in a high data 
incidence rate. 
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Furthermore, the memory requirement for the image data is 
substantial. To solve this problem there is, for example, a 
device known from WO 02/41031 for image data processing which 
5 preferably has two image sensors, each of which has a so- 
called Sample & Hold device. An image sensor of this type can 
be switched to the active state, to capture an image, for a 
definable image capture time. After an image has been 
captured, the image data for the image which has been 

10 captured, consisting of individual pixels, can be temporarily 
stored for a definable storage time directly on the image 
sensor chip in a memory, e.g. a capacitor. The image is so-to- 
speak "frozen" into the image sensor. An image sensor with 
this type of design is preferably constructed using CMOS 

15 technology, and is also called a "Synchronized Shutter" image 
sensor . 

The image data which is frozen in and stored on the image 
sensors is read out by a control device. The rate of read-out 
20 of the image data from the image sensors can then be adjusted 
to the processing speed of the microcontroller in the control 
device . 

The provision of images or image sequences in or outside a 
25 motor vehicle is made more difficult by a series of 

restrictions or conditions, as applicable. In particular, one 
or more image capture units must work reliably under all 
lighting conditions. A known way of achieving this is to 
provide, in addition to the image capture device, an 
30 illumination unit which, for the purpose of illuminating the 
image field which is to be captured, emits for example very 
intense and short IR (= infrared) light pulses with 
wavelengths lying in the near infrared range (> 800 nm) in 
order, for example, to avoid distracting or disturbing the 



-2- 



driver and passenger. This takes place, in particular, while 
the image sensors are switched to the active state. The 
negative effect of the additional light arising from the 
ambient lighting, for example as a result of strong sunlight, 
5 can thus be reduced if the image capture time is the same as 
or of the same order of magnitude as the duration of the light 
pulse. To achieve good overall illumination, a large number of 
IR LEDs are currently required. If necessary, a diffuser or 
specially calculated optics are placed in front of the LEDs. 

10 

By using the shortest possible exposure times, in the range 
for example of 50|Lls up to a maximum of 2ms, it is also 
possible to capture a sharp image of moving objects. The 
maximum permissible speed of the objects in the image field 

15 which is to be captured at which it is still possible to 

capture a sharp image depends on the maximum duration of the 
light pulse and the time gap between successive light pulses. 
In this situation, it is preferable if each light pulse 
illuminates one image. The illumination is then not 

20 uninterrupted, but is synchronized with the recording by the 
image capture unit, and is preferably only provided when the 
external light from outside is insufficient. The LEDs thus 
only emit light pulses, also called IR flashes, when it is 
necessary and at the exact time when the image capture unit is 

25 recording an image. These LEDs can thus realize the IR pass at 
the same time. 

Furthermore, the scene which is of interest must be well 
illuminated in spite of the limited installation space, and 
30 spatial arrangements which may be unfavorable, and the power 
loss or heat which results from the supplementary lighting 
must not damage the electronics of the complete system. 
However, if the illumination unit is accommodated in the same 
housing as the image capture unit with its image sensors and 
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the analysis electronics, this results in the electronic 
components warming each other, so that only a relatively low 
operating temperature can be achieved for the system. In the 
case of the heating problem, attempts are made to overcome it 
5 by good quality and generally high-cost cooling measures, such 
as LEDs on aluminum boards (FR4 on Alu, Flex on Alu, or the 
like) and by good quality components with low power loss and a 
high operating temperature range. 

10 Finally, an additional factor is that an illumination unit 
which has the same orientation as the image capture unit is 
not good for an image processing method which is primarily 
optimized for analyzing edge structures, because it casts 
hardly any shadows . 

15 

The object of the invention is to specify improved device as 
well as methods for capturing an object or a person, in 
particular in the passenger compartment of a motor vehicle, 
which significantly reduces the above-mentioned disadvantages. 

20 

This object is achieved by the characteristics of the 
independent claims. Advantageous embodiments and developments, 
which can be used individually or in combination with one 
another, are the subject of the dependent claims. 

25 

The invention for capturing an object or a person, for example 
inside the passenger compartment of a motor vehicle, with at 
least one illumination unit which emits light pulses for the 
purpose of illuminating an image field which is to be 
30 captured; and an image capture unit which incorporates at 
least one image sensor which takes in the light pulses 
reflected from an object or a person within the image field, 
and captures the image data for the object; in which at least 
one illumination unit is arranged to be spatially separated 



-4- 



from the image capture unit within or on the motor vehicle; is 
characterized by the fact that of the separately-arranged 
illumination unit and image capture unit, the one unit an 
optical transmitter, which emits control light pulses for the 
5 purpose of synchronizing or controlling the unit, as 

applicable, and the other unit an optical receiver which 
receives the control light pulses. This enables an 
advantageous choice to be made of the location, so that it is 
both favorable from the heating point of view and also 
10 favorable for illuminating the scene which is of interest - 

It is advantageous if a fiber optic cable is provided, 
arranged between the optical transmitter and optical receiver, 
for the purpose of transmitting the control light pulses. 

15 

Insofar as it is desirable to avoid a fixed cord, the laying 
of which can be extremely expensive, an alternative suggested 
for this purpose is the provision of transmission facilities 
on the optical transmitter and the optical receiver, for 
20 cordless transmission of the control light pulses. 

It is preferred in accordance with the invention that the 
optical transmitter is a component of the image capture 
unit. The advantage of this is that the illumination does 
25 not need to be uninterrupted, but can be effected 

synchronously with the making of records by the image 
capture device, in particular in relation to the exact 
moment when the image capture unit is recording an image. 

30 In order to achieve a high level of security against external 
sources of light pulses and against erroneous initiation of 
the illumination, it is advantageous if the control light 
pulse is transmitted in a form modulated by a signal or 
encoded by a suitable modulation method, as appropriate. 
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In accordance with the invention, control light pulses which 
have a wavelength lying in the near infrared range are 
preferred. The advantage of this is that it does not divert 
5 or disturb either the driver or the passenger or other 
occupants of the motor vehicle, or those in the traffic 
outside the motor vehicle. 

For the purpose of improving the casting of shadows, the 

10 illumination unit is, in accordance with the invention, 

preferably directed towards the person or object of interest, 
as applicable, and is arranged relative to the image capture 
unit at an angle a which, for the purpose of avoiding 
excessively large dark areas, for example on the face of an 

15 occupant, will preferably be from 0° to 45° or (in the 

possible situation where several illumination units are used, 
also) between 135^ and 180*^, whereby the latter range of 
angles can be used to exploit counter- lighting effects to 
advantage. Both arrangement variants, which could of course 

20 also be used in combination, bring the particular advantage of 
improved illumination. It is preferable if the power supply 
for the illumination unit(s) is independent of that for the 
image capture unit, so that the power loss is also moved 
completely away from the image capture unit, which is an 

25 advantageous way of minimizing the heating problems mentioned 
in the introduction. 

Finally, it is suggested that any possible timing offset 
between the time of illumination and the time of image 
30 recording is compensated by the control light pulses being 
emitted at an earlier point in time. 

With the present invention it is easier to deal in an 
advantageous way with conditions of limited space, such as are 
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typical for motor vehicle applications. In addition, the 
present invention permits better edge detection from the 
shadows which the illumination unit casts on structures and 
objects in the interior and/or the exterior of a motor 
5 vehicle. Finally, it permits easier adaptation to various 
"carlines" or vehicle types. The present invention is thus 
suitable not only for occupant protection systems in modern 
motor vehicles, but also for all motor vehicle related video 
systems using active IR illumination. 

10 

Additional details and further advantages of the invention are 
described below by reference to preferred forms of embodiment, 
in conjunction with the attached drawing. 

15 This shows, in schematic form: 

Fig. 1 device in accordance with the invention for capturing 
an object or a person in the interior space of a motor 
vehicle; and 

20 

Fig. 2 an enlarged detailed extract of the device in 
accordance with the invention shown in Fig. 1. 

Fig. 1 shows a device in accordance with the invention for 
25 capturing an object or a person 2, in particular in the 

interior of a motor vehicle 1. The device includes at least 
one image capture unit 6, which is arranged for example 
between the driver's seat 12 and the passenger's seat 13 in 
the roof structure of the motor vehicle 1. The image capture 
30 unit 6 has a working link to an illumination unit 3 which is, 
in accordance with the invention, arranged to be spatially 
separated from the image capture unit 6 in the motor vehicle 
1, for example in the roof structure adjacent to the A-post, 
or in the latter, or in some other suitable place such as on 
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the rear-view mirror, beside the windscreen or some such 
location (not shown) . To improve the casting of shadows, the 
illumination unit 3 is directed at the person or object 2, as 
applicable, and relative to the image capture unit 6 is 
5 arranged at an angle of a, which when the counter-lighting 

effect is being used can preferably be 135° to 180*=", and which 
in Fig. 1 has, for example, been chosen as 140°. Insofar as it 
is useful, further illumination units (not shown) can also be 
provided- It is preferable if the power supply for the 
10 illumination unit(s) 3 is independent of that for the image 
capture unit 6 . 

For the purpose of synchronizing or controlling, as 
applicable, the units 3 and 6 there is a fiber optic cable 11, 

15 for example, arranged between the image capture unit 6 and the 
illumination unit 3 (shown on the left-hand side of Fig. 1) . 
Ah alternative to this, and particularly preferred, is 
cordless synchronization between the units 3 and 6 effected 
(as shown on the right-hand side of Fig. 1) with the help of 

20 suitable facilities 8 and 9 for transmitting control light 
pulses 10. 

The image capture unit 6 includes at least one image sensor 7 
(shown on the right-hand side of Fig. 1) . Preferably, however, 

2 5 image capture units 6 are provided with two image sensors 7 

(shown on the left-hand side of Fig. 1) . Image capture units 6 
of this type are also called stereo cameras. The optical axes 
of the image sensors 7 of a stereo camera 6 are a definable 
distance apart and it is possible in an advantageous way to 

3 0 record a stereo image or a 3-D image, as applicable, of 

movable objects 2, for example, and to communicate them to a 
microcontroller (not shown) for further processing. With the 
help of algorithms, it is then possible to determine the 
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distances from the objects or persons 2 to the image sensors 
7. 

Fig. 2 is based on Fig. 1, and shows an enlarged detail view 
5 of the device in accordance with the invention. For the 

purpose of improving the casting of shadows, the illumination 
unit 3 is aligned towards the person or object 2, as 
applicable, and relative to the image capture unit 6 it is in 
turn arranged at an angle of a, which, as an alternative or 

10 addition to Fig, 1, can preferably be 0*=» to 45®, and in Fig. 2 
has been chosen by way of example as 40°. Preferably, the 
illumination unit 3 will alone be capable of emitting light 
pulses 4 for the purpose of illuminating an image field 5 
which is to be captured. However, it is noted that 

15 combinations of the arrangement in accordance with the 

invention of image capture units 6 and illumination units 3, 
even with familiar image capture device, are not excluded. In 
particular - depending on the vehicle type or special 
application - further illumination units (not shown) may be 

20 useful in or on the motor vehicle 1, as appropriate, and/or in 
or adjacent to the image capture unit 6. It should be 
emphasized in addition that the use of the present invention 
for vehicle-related so-called external applications can also 
be appropriate. 

25 

It can also be clearly seen in Fig. 2 how, in accordance with 
the invention, the image capture unit 6 has an optical 
transmitter 8 which emits control light pulses 10 for the 
purpose of synchronizing or controlling, as applicable, the 
30 units 3 and 6. Correspondingly, the illumination unit 3 has an 
optical receiver 9, which receives the control light pulses 
10. Cordless transmission of the control light pulses 10 is 
effected by means of an optical transmitter 8 or optical 
receiver 9, preferably using control light pulses 10 with a 
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wavelength lying in the near infra-red range (IR) (> 800 nm) 
which, for the purpose of avoiding mistriggering or to protect 
against external sources of infra-red like that of the 
illumination unit 3, or from locking and unlocking systems or 
5 the like, are preferably transmitted in modulated and/or 

encoded form. If there should be any propagation time delay 
between the image recording and illumination which needs to be 
compensated, this can be effected by earlier emission of the 
control light pulses, for example. 

10 

With the present invention, it is possible to deal with 
restricted space conditions in or on a motor vehicle 1 more 
simply and in an advantageous way. In addition, the present 
invention permits improved edge detection by the illumination 

15 unit 3 casting shadows on the structures and objects 2 inside 
or outside a motor vehicle 1. Finally, it permits simpler 
adaptation to different "carlines" or vehicle types. The 
present invention is thus suitable not only and particularly 
for occupant protection systems but for all motor vehicle 

20 related video systems using active IR illumination. 
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